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1. Introduction 


Fellegi (1996) provides a strong argument that the trust in the national statistical 


agency is how most users judge the quality of its statistical products. 


"Credibility plays a basic role in determining the value to users of the special 
commodity called statistical information. Indeed, few users can validate directly the 
data released by statistical offices. They must rely on the reputation of the provider 
of the information. Since information that is not believed is useless, it follows that 
the intrinsic value and useability of information depends directly on the credibility of 
the statistical system. That credibility could be challenged at any time on two 
primary grounds; because the statistics are based on inappropriate methodology, or 


because the office is suspected of political biases." 


Trust will not happen unless the culture is right. Culture is a word with many 
meanings but | am interpreting culture as "the way we do things". Core values are 
important to this. They cannot be just statements hanging on the wall. They have to 
be understood. They have to be reflected in behaviours, particularly by leaders of 


organisations. 


The Australian Bureau of Statistics (ABS) places great reliance on adherence to its 
core values. More than anything, they distinguish us with other survey providers. 


The core values are: 


e Relevance - regular contact with those with policy influence, good statistical 
planning, which requires a keen understanding of the current and future needs 
for statistics, are essential, as is the need for statistics to be timely and relatable 


to other statistics. 


e Integrity - our data, analysis and interpretation should always be objective and 
we should publish statistics from all collections. Our statistical system is open to 


scrutiny, based on sound statistical principles and practices. 


e Access for all - our statistics are for the benefit of all Australians and we ensure 


that equal opportunity of access to statistics is enjoyed by all users. 


e Professionalism - the integrity of our statistics is built on our professional and 
ethical standards. We exercise the highest professional standards in all aspects 
of ABS statistics. 


e Trust of providers - we have a compact with respondents; they are to encourage 
us to provide accurate information and we ensure that the confidentiality of the 
data provided is strictly protected. We keep the load and intrusion on 
respondents to a minimum, consistent with meeting justified statistical 


requirements. 


Adherence to core values is just one element of maintaining a quality culture. Part 2 


discusses the steps the ABS uses to maintain a quality culture. 


It is now widely recognised that quality is much more than accuracy (eg 
Brackstone(1999) and Carson (2000)). In Part 3, | will discuss the different 
dimensions of quality before identifying in Part 4 what | think are some of the major 
quality challenges for the Australian Bureau of Statistics (ABS) over the medium 
term. | believe that most of these will be shared by other national statistical 


organisations. 


2. Towards a High Quality Statistical Service 


Quality assurance is a responsibility of all staff in the ABS. There is no central 
"quality management" group although Methodology Division is encouraged to be our 
conscience on quality issues - a role it takes on with enthusiasm, sometimes to the 


annoyance of others. However, | see that as a good sign - they are provoking 


debate on some of the more difficult quality issues. Support from senior 


management for this type of role is very important. 


The key strategies to ensuring a high quality are described under six broad 
headings. 


e Ahigh degree of credibility for the ABS and its outputs. 
e Maintaining the relevance of ABS outputs. 

e Effective relationships with respondents. 

e Processes that produce high quality outputs. 

e Regular review and evaluation of statistical activities. 


e Staff who are skilled and motivated to assure the quality of ABS outputs. 


2.1 A High Degree of Credibility 


Credibility is fundamental to the effective use of official statistics. Credibility arises 
from a system of statistics which provides an objective window upon the condition of 


a nation's economy and society. 


The legislative framework within which the ABS operates is an important 
pre-condition for the integrity of Australia's official statistics. The Australian 
Statistician is guaranteed considerable independence by law. This helps ensure that 
the ABS is, and is seen to be, impartial and free from political interference. In 
particular, the independence of the Statistician supports his objectivity in determining 
the statistical work program. Although the legal authority is there, it still needs to be 


reflected in the way my senior staff and | behave. 


Government statisticians must not just apply professionalism skills to their work; they 
must also be seen to adhere to high ethical standards, especially with respect to 
objectivity and integrity. We are frank and open when describing our statistical 
methods to users; we publish information about our performance - for example, in 
terms of both sampling and non-sampling errors, and revision histories for key 
series; we are willing and able to identify and address user concerns regarding 


quality; we are receptive to objective criticism and be prepared to respond quickly 


even if the problem is one of perception rather than reality. Also, we promote good 
relationships with the media as they have a major influence on public opinion of the 
ABS and its outputs. Also, most Australians find out about official statistics through 
the media. We engage in other user education activities aimed at fostering 


intelligent use of official statistics. 


The fact and perception of ABS objectivity are reinforced by our policies of 
pre-announcing publication dates for main economic indicators, allowing very limited 
pre-release of publications (the details of which are in the public domain), and 


making special data services available on an even handed basis to all. 


2.2 Maintaining the Relevance of ABS Outputs 


There is, of course, some tension between (on the one hand) being responsive to 
changing policy needs and (on the other) maintaining the continuity of a system of 
statistics that can objectively monitor performance. Senior staff of the ABS devote a 
great deal of attention to maintaining personal contact with key users, so our 
organisation can gather intelligence about policy issues and emerging areas of 
economic, social and environmental concern. This includes regular lunchtime 
meetings with the most senior staff of the government agencies responsible for 
policy. The Directors of our State offices have similar arrangements with State 
officials. That intelligence feeds into strategic planning and judgements when our 


national statistical programs are designed and reviewed. 


The ABS has a range of other means for communicating with the users of statistics, 
to ensure that our products are relevant to their needs. For example, advisory 
groups representing users and experts in various fields provide valuable guidance to 


our statistical activities. 


There may also be some tensions or trade-offs between the different aspects of 
quality. The ABS positions itself at the higher accuracy end of the information 
market, to protect the valuable ABS ‘brand name’. But if, for example, there is an 
urgent demand for data in a new field, some aspects of quality may be traded off in 
order to achieve timeliness and relevance. Nevertheless, there is a "bar" below 
which we will not go. Because it is probable that the new statistics will be used to 
inform significant decisions or debate, the ABS makes very clear statements about 
the accuracy of the data and the uses for which they are suitable. On occasion, 
such new statistics may be differentiated from our other products by labelling them 
‘experimental’ or releasing as an information or occasional paper. We regard this 


form of branding as very important. 


2.3 Effective Relationships with Respondents 


An official statistical agency must maintain good relations with respondents if it 
wants them to co-operate and provide high quality data. The ABS approach 
includes - explaining the importance of the data to government policy, business 
decisions and public debate; a policy of thoroughly testing all forms before they are 
used in an actual survey; obtaining the support of key stakeholders; minimising the 
load placed on respondents particularly by using administrative data where possible; 


and carefully protecting privacy and confidentiality. 


The ABS monitors and manages the load it imposes on both households and 
businesses; we have developed ‘respondent charters' for both groups. As well, a 
Statistical Clearing House has been set up within the ABS to coordinate surveys of 
businesses across government agencies (including the ABS), to reduce duplication 


and to ensure that statistics of reasonable quality are produced. 


All ABS forms and collection methods are tested to ensure that the data we seek are 
available at reasonable cost, and the best available ways are used to collect them. 
For business surveys, our units model, classifications and data items, are designed 
to be as consistent as possible with the way businesses operate. This now 
corresponds closely with reporting for taxation purposes, making it easier to 
integrate survey data with data collected for taxation purposes. For household 
surveys, the extensive use of cognitive testing tools within the ABS, and the 
establishment of a questionnaire testing laboratory, have helped to improve quality 
and to reduce respondent load. Standards for form design and form evaluation are 


set out in manuals and are promoted and supported by experts in form design. 


The ABS uses efficient survey designs to minimise sample sizes and hence total 
reporting load; we also control selection across collections to spread the load more 
equitably. To take advantage of the current reforms of the Australia taxation system, 
the ABS is seeking every opportunity to improve the efficiency of our sample 
designs, and to use taxation data as a substitute for the data now gathered through 
direct collections. As mentioned above we have started the process of simplifying 
the business unit structure used in our surveys to make it consistent with the 


structure used for taxation purposes. 


For household surveys, the introduction of computer aided interviewing has helped 
to streamline interviewing procedures, reduce respondent load, and improve the 


quality of data collected. 


2.4 Processes that Produce High Quality Outputs 


The quality of ABS statistics is underwritten by the application of good statistical 
methods during all stages of a collection including the design stages. The ABS has 
a relatively large Methodology Division (about 120 staff) which reports directly to the 
Australian Statistician. The Division is responsible for ensuring that sound and 
defensible methods are applied to all collections and compilations. Also, the 
Methodological Advisory Committee, a group of academic experts, provides 


independent reviews of our statistical methods. 


The ABS puts substantial effort into developing statistical standards, including 
concepts, data item definitions, classifications, and question modules. All ABS 
surveys must use these standards. The standards are supported by relevant data 
management facilities to ensure they are accessible and to make it easier to use 


standard rather than non-standard approaches. 


Sample design and estimation methods are the responsibility of the Methodology 
Division. Our designs aim to minimise cost for adequate accuracy. Where possible, 
a "total survey design" is used - accuracy requirements are set according to the 
intended use of the data, and accuracy is measured in terms of both sampling and 
non-sampling errors. In our business surveys for example, total survey design 
guides the allocation of resources to the intensive follow up of non-respondents or 
the editing of questionnaires; the effort for reducing non-sampling errors is optimised 
according to the impact of errors on overall quality. The cost to data providers is 
also taken into consideration. The "total survey design" has to be approved by a 


senior ABS staff member before it is implemented. 


In recent years, the ABS has made substantial progress by applying standardised 
best practice across surveys. For example, business surveys based on the business 
register now draw their frames at a common date each quarter, and uses a common 
estimation method to ensure all collections have a consistent and complete 
coverage. Standards rules are adopted for frame maintenance, field collection and 
estimation, and generalised processing facilities are available to support the use of 
these rules. Standard methods are used to allow for "new businesses" not yet 
included on the survey frame. The ABS is thereby able to ensure the coherence of 


estimates across different business surveys. 


For household surveys, a master sample system has been adopted since the mid 
1960's. The system is updated regularly after each five-yearly census, and has 
been the cornerstone for ensuring the accuracy of statistics collected from 


household surveys. 


Achieving quality in surveys is easier when computer systems support current best 
practice. The ABS has invested in generalised tools for collection and processing. 
Generalised tools have been developed for all major processing steps of both 

business and household surveys, including sample frame management, data input 


and editing, imputation, estimation and aggregation. 


The ABS embraces a rigorous continuous quality improvement approach wherever 
appropriate. The Australian Population Census is a classic example of raising 
quality through a strategy of measuring quality and involving all staff in examining 
and devising solutions for quality problems. This approach was applied very 
effectively at the data processing centre for the 1996 Census; the centre achieved 
significant savings, better quality and an improvement in timeliness. Continuous 
quality improvement is also applied to the coding of businesses on the business 


register, and to many other ABS processes. 


At the output end of collections, each subject group is required to confront its data 
with other ABS data and with external information, to ensure the coherence of our 
statistics. Most of the key macroeconomic have to be "signed off" by the national 
accountants in meetings established especially for the purpose of clearing the 
statistics. The national accountants then have an obligation to use this data without 
further adjustment in the compilation of the accounts, enhancing consistency. More 
generally, confrontation of different data sources is undertaken by our national 
accountants Since 1998, the ABS has adopted an ‘input-output approach’ to 
compiling national accounts estimates, and the new methodology has led to more 
consistent accounts. In particular, the rigorous data confrontation and balancing 
process at detailed levels have helped to identify data deficiencies and have 
suggested methods for reducing discrepancies between estimates. Information 
about quality is fed back to the economic collection groups and is resulting in a more 


focused approach to improvements in the quality of source data. 


One important quality improvement initiative that the ABS has pursued is the 
development of an Information Warehouse to manage and store all of our 
publishable data. By drawing together different datasets into a single database, the 
Warehouse enables our statisticians to confront statistics produced from different 
collections. Furthermore, as all forms of publication, be they paper based or 
electronic, are to be produced from a single data store, with the objective of 
ensuring that data released in different products, and at different times, are 


consistent. 


Another important element of quality management is documentation. Good 
documentation supports review activity and facilities the dissemination of quality 
information to users, so they can assess the fitness of the data for the purposes they 
have in mind. As part of the Information Warehouse initiative, the ABS can now 
enforce standards for documentation of the metadata that describe concepts, 


definitions, classifications and quality. 


A relevant and responsive statistical service must do more than provide data to 
clients. The ABS has also recently strengthened its analytical ability. A team of 
analysts has been set up to develop new measures of socioeconomic concepts, to 
explore relationships between variables and to prototype new analytical products. 
The expanded program of analysis work is also expected to deliver significant 


benefits in the form of insights into data gaps and quality concerns. 


2.5 Review and Evaluation of Statistical Activities 


Each ABS area is responsible for continuous quality review and improvement. For 
statistical collection areas, quality management is supported by sets of performance 
indicators. A standard set of measures has been developed to permit a comparison 
of quality across collections. Tools are now being developed to calculate these 
measures as part of our normal survey processes, and the Information Warehouse 
will allow us to store and display the measures. The key indicators are also included 


in the annual reports each Branch makes to the ABS Executive for review. 


Quality measures are also of interest to the users of statistics. The Information 
Warehouse will improve users' access to information about quality issues. As well, 
the ABS places high priority on helping users understand the quality of data and its 
implications for them, and has adopted active education strategies to promote such 
understanding. As highlight in Lee and Allan (2001), there is much to do to improve 


user understanding of quality. 


Each ABS household survey now includes an evaluation program which reviews the 
effectiveness and efficiency of all survey activities and assesses the extent to which 
the data are used by clients. The Statistical Clearing House conducts a review of 
each ABS business survey. These initiatives ensure that all collections are 
subjected to at least a basic evaluation, and brings to light opportunities for 


improvements to our quality and efficiency. 


As well as making internal comparisons of performance across its own collection 
areas, the ABS has established a benchmarking network with overseas statistical 
agencies; the aim of the network is to share information about survey design, 
processes and costs. The benchmarking exercise is providing invaluable guidance 


to the ABS's efforts to improve its processes and outputs. 


Finally, each subject matter area and support area in the ABS provides an annual 
report to the senior management forum, highlighting progress and problems in the 


area and providing performance measures. 


2.6 Skilled and Motivated Staff 


The ABS could not provide high quality information to its user community if it did not 
employ people who bring skills and energy to our statistical work. The staff are 
responsible for implementing the strategies discussed above. They must take a 
professional approach and be committed to the development of new methods, to 
continuous quality improvement, and to the open discussion of methods and quality 


issues. 


Quality improvement and on-going statistical work compete for the time and energies 
of our staff. The ABS's approach is, as far as possible, to integrate quality work with 
on-going processes and systems. We also emphasise to staff that quality 
management is a corporate priority and ensure that tools and resources are made 
available to support it. In particular, the ABS is implementing a tighter approach to 


project management; this is being supported by manuals, systems and training. 


Statistical training plays an important role in maintaining and improving quality. The 
ABS is always searching for new, more effective, approaches to skills development. 
An important element of our performance management system is a focus on 


identifying and addressing individuals' development needs. 


Relationships with other national and statistical agencies are also a very important 
element of the ABS's efforts to improving official statistics. The ABS is committed to 
using international standards; we take advantage of the wide range of expertise 
embodied in those standards. On the other hand, there is an obligation for us to 
make a positive contribution to the development of the standards. In doing so, we try 
to take account of the interests of the Asia/Pacific region as well as those of 
Australia. With ever increasing globalisation of economic activity and the pursuit of 
world wide social goals, the compatibility between Australian's statistics and those of 
other countries, is an important element of quality. The ABS maintains strong links 
with many overseas agencies. We are fortunate that there is a lot in common in the 
challenges we face and there are great benefits from sharing experiences with other 


statistical agencies. 


Se Dimensions of Quality 


Figure 1 is taken from Lee and Allen (2001). Among other things, it neatly 
summarises, on the left hand side, three existing frameworks for judging quality. 
There are some differences with the descriptors used but basically they are 
providing the same message - there is much more to quality than accuracy. This is 
now widely accepted although it was not so long ago that discussion of the quality of 


a Statistic focussed on its accuracy and the sampling variability in particular. 
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There are also several messages in the right hand side of Figure 1. 


(i) 


There are many different ways of compiling official statistics - from modelled 


data/analytical outputs to sample surveys. The quality challenges differ 


between the different means of compiling statistics and in Australia we are 


making greater use of administrative data, systems of accounts (linked to the 


national accounts) and models or analytical methods to produce statistical 


outputs, compared with five years ago. 


. 


(ii) There are several groups of activities associated with statistical outputs - from 
"frameworks, concepts, standards and classifications" through to 
"services/dissemination". Each is important in its own right and has its own 


quality challenge. 


(iii) © The Performance of a National Statistical Office is extremely important to its 
quality image as recognised in the opening quote of the paper. A number of 
the elements are specified in Figure 1. All are important. Indeed you cannot 
have a high performing statistical office unless you rate well against each of 


these elements. 


(iv) | There are other elements such as institutional settings which are also an 


important element. 


My main purpose is describing the above is to emphasise that the list of quality 
challenges for a national statistical office is very large. All have to be tackled in some 
way - this would not be possible unless you have a quality culture ie attention to 
quality is the responsibility of all staff. There are many "moments of truth" to 


genuinely test the quality culture. 


Psychologists say that it is not possible to grasp more than seven points at one time 
so | have limited myself in the remainder of this paper to identifying seven major 


quality challenges for the ABS. 


(i) The increasing use of large, but imperfect, administrative and transactional 


data bases for compiling official statistics. 


(ii) Increasing user expectations are raising the quality "bar". 


(iii) | Managing the tension between improving business processes (which often 
means removing those responsible for statistical outputs from direct 
involvement with input processes) and maintaining or improving the quality of 


statistical outputs. 


(iv) | Quality assurance on electronic outputs. 


(v) | The presentation of statistics on the internet, including the need to educate 


the user community on quality of official statistics. 


(vi) Managing the transfer of knowledge and skills with an ageing senior 


management team, many of whom will retire over the next 5 years. 


(vii) Use of international statistical standards to maintain comparability where the 


standard may not be the most appropriate for national statistics. 


4.1 Increasing Use of Administrative/Transactional Data Bases 


We have used administrative data bases for many years (eg vital registrations for 
births and deaths, customs for trade data) to compile office statistics. Others have 
been used to develop frameworks for statistical collections. The issues at hand are 
the increasing availability of these data bases, their under-utilisation for statistical 
purposes, and taking advantage of the potential to link across data bases and ABS 
data sets using a common identifier (ie the Australian Business Number for business 


statistics). 


Examples of administrative data bases that are becoming available are extended 
personal and business income tax data bases, health insurance transactions, and 


details of those on income support. 


Transactional data bases are becoming available, although not in readily accessible 
form. Data bases of particular interest to the ABS are scanner data bases from retail 


outlets and eftpos data bases. 


There are some particular advantages in using administrative or transactional data 


bases: 


e they reduce the compliance cost we impose on respondents 


e they are often "censuses" and therefore provide scope for producing detailed 


data sets (eg by geography) 


e they often have a longitudinal element (eg tax data) to support this form of 


analysis 


e they often contain an identifier which facilitates analysis across data sets (eg the 
Australian Business Number will facilitate analysis across business tax data sets, 


customs data, and ABS surveys). 


There are negatives of course - for example, the definitions may not be consistent 
with statistical concepts; less attention may have been given to incoming quality; and 
they may be out of date. Managing privacy aspects is a particularly important 
element. Although are motives are entirely honourable, matching data bases is a 
sensitive issue and ignored at our peril. Many of our users, particularly those in the 


academic community, are not as sensitive to these concerns. 


There is also the question of whether the ABS should produce the statistical outputs 
or the agency responsible for the data sets. A number of issues come into 
consideration - the importance of the outputs to the national statistical service, costs, 
the extent to which quality can be managed and the basic question of whether the 
administrative agency is prepared to give up custodianships. Only the most 
important data sets will be brought into the ABS for compiling official statistics; for 
the others, we will work with the administrative agency to help them deliver statistical 


outputs into the public domain. 


What have been our key responses to this important quality management issue? 


e Weare developing protocols for the publication and management of data from 
administrative sources. Associated with this is the promotion and support of 


good statistical and data management practices. 


e For each statistical field, we are preparing information development plans which 
identify those areas of greatest importance and set out specific activities which 
will lead to increased availability of non-ABS data, particularly quality 


management issues. 


e A major investment project has been the greater utilisation of taxation data to 


provide cost-effective statistics. 


e We are investigating methods for assuring the quality of the very large but 
imperfect data sets that are available through administrative and transactional 


data holdings. 


4.2 Increasing User Expectations 


User expectations on quality are changing - they are much higher than what they 
were as recently as 5-10 years ago. This trend is likely to continue. The increasing 
globalisation of financial markets will mean that key macroeconomic statistics have 


international, as well as national prominence. 


There is a perception that statistics have become more volatile. In some cases they 
have because the underlying phenomenon has become more volatile. However, we 
don't believe statistical measurement methods are a significant contributing factor - 
in most cases development methods have led to improvements although the 
perception may be different. For example, the volatility in the key national accounts 
series is considerably less than what it was 10-15 years ago yet this is quite different 


to the perception of many users. 


We also receive more criticism of inaccuracies in very detailed data (eg Population 
Census tables) than previously. Again, it is not that the quality is deteriorating - it is 


the expectation is higher. 


We have to accept that "the bar is rising" and do what we can to improve quality to 
the expected level. That is not always possible of course so managing expectations 


is important. This can be done by: 


providing good explanations of the strengths and weaknesses of particular data 


sets; 


e talking to key users whenever possible about the strengths and weaknesses of 


data series; 


e responding to their informed criticism (Seek partnerships in improving quality eg 
in our detailed foreign trade statistics we openly seek feedback from users on 


the quality of the statistics); and 


e providing as much explanation as possible for statistics that might seem unusual 


or different to expectation. 


4.3 Improving Business Processes 


Like several statistical organisations, the ABS is looking at how it might use new 
technologies, and other elements such as increased access to taxation data, to 


improve the efficiency of its business statistics processes. 


We are also investigating the business processes associated with household 
surveys, particularly as increased use is made of CAI. However, in this section | will 


concentrate on business statistics to describe this particular quality challenge. 


A team has been set up to look at the possibilities - referred to as strategic 
scenarios. We are looking for revised business processes that will be in place for at 
least 10 years and will yield a significant return on the investments required to set it 


up. It is likely that we will: 


e extend the responsibilities of the Business Register Unit to capture and store 
taxation data with a direct link to the Business Register through the Australian 
Business Number (ABN). The ABN is now allocated through the taxation 
registration scheme and is available with most business transaction data bases 
(It is also collected in ABS surveys.).The data is stored in a way that it can be 
used by the various ABS statistical areas to compile statistics directly from 
taxation data or in combination with ABS survey data; 


e setup anew business statistics processing environment based around an input 
data warehouse, with the Australian Business Number as the link across the 


various data sets; 


e there will be greater centralisation of a number of the functions associated with 


compiling business statistics. 


We can see the positives in these developments - more efficient delivery of business 
statistics, enhanced use of taxation data and other administrative data, data bases 
that support a wider range of statistical analysis. However, it will reduce the level of 
contact that statistical output areas have with their input data sources. What impact 
will that have on quality? What strategies can we deploy to mitigate the impact? 
These are important questions that we will have to answer before making a final 


decision on how we are to manage our business statistics in the future. 


4.4 Quality Assurance on Electronic Outputs 


Great care is taken on the quality of our paper products. This has been built on 
many years of experience. Our record is good and the quality assurance processes 
well embedded in the way we go about our business. Yet, more and more of user 
community receive their data in electronic form only. They will make analyses based 
on these outputs often leading to important decisions being made. It is just as 
embarrassing to us to have errors in electronic outputs as to have them in paper 


outputs. 


Our quality assurance procedures for electronic outputs are not as sophisticated, but 


they are evolving. The key responses have been as follows: 


e Our data warehouse supports the storage of all the objects associated with the 
dissemination with a particular set of statistics, including data cubes and meta 


data. 


e Statistical areas are asked to approve each object - they are individually 
developing their own techniques for quality assurance (but sharing ideas on best 


practice). 


e A publishing system is being developed which supports the simultaneous 
release of all outputs. If they are delivered from the same set of objects, there is 


less chance of inconsistency between the outputs. 


4.5 The Presentation of Statistics on the Internet 


Ultimately the user can only make judgements about the fitness of a statistical output 
for their purposes. These vary of course and what might be fit for one purpose may 
not be for another. There is an obligation on us to provide a range of supporting 
information on data outputs, including that on quality, so that they can make their 
own judgements on fitness of use. There are a number of existing, well proven 
practices relating to declarations about the quality of statistics. These activities are 


now a routine part of existing dissemination practices. They include: 


e Concepts, Sources and Methods publications which describe in detail the 
methods used to compile major statistical outputs. These are available on our 


web site as well as on other media. 


e Anassortment of Information and Working Papers, and feature articles in 
publications, which are used to draw attention to issues specific to particular 


outputs or changes that are being made to their compilation methods. 


e Apolicy of "no surprises" when there are significant changes to the methods 
used for the compilation of statistical series. As well as Information Papers etc, 
if there are important changes to statistical series, we embark on a program of 
seminars and bilateral discussions with key users to explain the changes and the 
reasons for their changes. 


e Material on methods is included in all our publications. The ordering and physical 
presentation of this information is according to agreed standards. These were 
developed following research undertaken for us by a communications consultant 


on how our users use the material in statistical publications. 


e The analysis section of our publications includes material which explains, among 
other things, large or unusual movements in our statistical series. Often this will 
be based on information that is only available to ABS staff through their contact 
with respondents or their intimate knowledge of the methods used in compiling 
statistics. Our User Groups have advised that this is one of the most valuable 


forms of analysis that we can undertake. 


We believe that our key users have a reasonable understanding of the quality of the 
statistics they use. However the increased reliance on electronic dissemination 
poses new challenges. In one sense this move provides a wonderful opportunity to 
present a range of information on quality that is easily accessible through a few well 
designed "clicks". But because information about the quality of the statistics is "not 
in your face" like it can be in hard copy publications it is easier for users to avoid the 
key messages that your are trying to convey. The real challenge for us is to develop 
methods for presenting quality in a way that is not easy for users to avoid the main 


messages we want to convey. 


One means of doing this may be to provide separate messages which draw their 
attention to particular information you want them to receive on quality. These could 
be automatically activated as particular statistical series are accessed or could be 
delivered by a separate email message. Research is required into the most effective 


means. 


Lee and Allen (2001) have described some of our research work to date on this 
issue . The work is still at the exploratory stage. Things that are being investigated 


are: 


e Useability testing of how users prefer to access information on quality. 


e Showing leadership and developing of user education programs on how to use 


information on quality. A trial version of the course will be available in late 2001. 


e The development of four prototype tools to assist users understand the quality of 
particular statistics. The four prototype tools are "Quality Issue Summaries", 
"Quality Measures", "Data Accuracy" and "Integrated Access to Data and 


Metadata". 


| have not gone into detail on these developments as Bill Allen will be giving a 


separate presentation on this work later in the Conference. 


4.6 Managing the Transfer of Knowledge and Skills 


Like several other national statistical organisations, many of the ABS management 
team, and other senior staff, are aged in their 50's. Some have retired in recent 
years. Others are expected to over the next few years. If managed correctly, this is a 
great opportunity to refresh the organisation through providing new blood to 
management positions. These will normally be younger staff who will bring new 


ideas and energy into the management team. 


On the other hand, experience and know-how will be lost. Both sides of this 


equation need to be managed carefully. Our strategy is as follows. 


e We have developed special programs for those staff with potential. Specifically, 
they undertake a leadership and management development program which has 
been specially customised for the ABS. Staff are chosen for these programs by 


senior managers. You cannot self select yourself to be a participant in the 


program. Furthermore, after staff have completed the program they can be 
expected to be chosen for a special assignment or rotated to a new position. 
The underlying philosophy is that the best way of learning is to obtain a variety of 
work experiences. A very high proportion of recent promotions to senior 
management positions have been participants in these programs. So far this 
has helped us to adequately cover the gaps created by a larger number of 


retirements than in the past. 


e We retain links with retired ABS staff through a variety of informal and formal 
means (eg social functions, including them on the distribution list for ABS News, 


etc). Their knowledge is accessible if required. 


e A stronger emphasis on knowledge management, using the facilities of our 
groupware product (Lotus Notes), means that key parts of our work are well 


documented and easily accessible. 


e The strong move to standardisation of methods and systems means there is less 


dependence on local knowledge. 


e Forsome key positions (eg Director of National Accounts) there is shadowing of 


work prior to the retirement of the incumbent. 


To date we have managed this transition well. We have been able to adequately fill 
vacant senior positions and at the same time refresh the organisation by promoting 


staff with fresh ideas. There is a need to remain adroit. 


4.7 Use of International Standards 


Our starting position is at where international standards exist we should use them. 
This has not always been the case. For example, although our industrial 
classification has been loosely based on ISIC, and a concordance developed with 
ISIC, the classification is largely homegrown reflecting the specific interests of 
Australia and New Zealand. We have agreed to use the 2007 version of ISIC, at 


least to the 3 digit level, with variations only where there are specific circumstances 


which justify it. 


There are often pressures on us to divert from international standards. Sometimes 
this is to make the Australian situation look better (eg government finance statistics). 
In other cases, such as with the ILO unemployment definition, the pressure is 
because the international definition does not seem to reflect the real situation in 
Australian circumstances. We resist these pressures but it is important that we have 
a well documented international standard as a reference point to justify our position. 
Nevertheless, diversions from the international standard are made on an exception 
basis - they need to be well documented with a clear explanation of the reason. In 
cases where there is a clear need to have information on a basis other than the 
international standard our position is that we should publish statistics on both basis. 
The headline figure would still reflect international standard as increasingly the 
Australian situation is being compared with that of other countries and it is important 


that it is done on a comparable basis. 


There is a tension that needs to be managed but if we are serious about the 
importance of international comparisons it is imperative that international standard is 
the main guiding light in developing the concepts, sources and methods used in 
Australia. There is also an obligation on us to make a significant contribution to the 


development and revision of international standards. 


5. Conclusion 


We would all agree that attention to quality is a fundamental aspect of our operation. 
In this paper, | have attempted to show that there are many dimensions to quality. 
This same message is clear from the frameworks for quality that have been 
developed by other organisations, such as the IMF, Statistics Canada and Statistics 
Sweden. The consequence is that a quality organisation depends on the actions of 
all it's staff as all can have an impact on quality in one way or another. It cannot be 
left to a project team designated as having responsibility for quality. Therefore, it 
can only happen if there is a genuine quality culture within the organisation. In this 
paper | have attempted to describe how we achieve this within the ABS. 


Nevertheless, it is important to have someone who performs the role of the corporate 


conscience on quality. Among other things they draw attention to the most important 
risks to quality or behaviours they see as contrary to our corporate objective. We 
have given this responsibility to Methodology Division and made the chief part of the 


Executive so that it is easier for key messages to be conveyed. 
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